System and method for integrated surgical table motion

ABSTRACT

A computer-assisted device includes a first articulated arm and a control unit coupled to the first articulated arm. The first articulated arm is configured to support an end effector. The control unit is configured to determine a virtual coordinate frame, the virtual coordinate frame being of an imaging device for capturing images of a workspace of the end effector, and the virtual coordinate frame being detached from an actual imaging coordinate frame of the imaging device; receive, from an input control configured to be manipulated by a user, a first instrument motion command to move the end effector; and drive the first articulated arm to move the end effector relative to the virtual coordinate frame based on the first instrument motion command.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/809,471 filed Mar. 4, 2020, and entitled “System and Method forIntegrated Surgical Table Motion”, which is a continuation of U.S.patent application Ser. No. 15/522,261 filed Apr. 26, 2017, and entitled“System and Method for Integrated Surgical Table Motion”, which is aU.S. National Stage patent application of International PatentApplication No. PCT/US2015/057673 filed on Oct. 27, 2015, and entitled“System and Method for Integrated Surgical Table Motion”, the benefit ofwhich is claimed, and claims priority to U.S. Provisional PatentApplication No. 62/134,292 filed Mar. 17, 2015 and entitled “System andMethod for Integrating Surgical Table Motion”, and U.S. ProvisionalPatent Application No. 62/069,245 filed Oct. 27, 2014 and entitled“System and Method for Integrated Surgical Table”, each of which ishereby incorporated by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates generally to operation of devices witharticulated arms and more particularly to repositioning articulated armswhen moving a patient and allowing instrument control duringdisturbances to the articulated arms and a moving endoscopic viewreference frame.

BACKGROUND

More and more devices are being replaced with autonomous andsemiautonomous electronic devices. This is especially true in thehospitals of today with large arrays of autonomous and semiautonomouselectronic devices being found in operating rooms, interventionalsuites, intensive care wards, emergency rooms, and the like. Forexample, glass and mercury thermometers are being replaced withelectronic thermometers, intravenous drip lines now include electronicmonitors and flow regulators, and traditional hand-held surgicalinstruments are being replaced by computer-assisted medical devices.

These electronic devices provide both advantages and challenges to thepersonnel operating them. Many of these electronic devices may becapable of autonomous or semiautonomous motion of one or morearticulated arms and/or end effectors. These one or more articulatedarms and/or end effectors each include a combination of links andarticulated joints that support motion of the articulated arms and/orend effectors. In many cases, the articulated joints are manipulated toobtain a desired position and/or orientation (collectively, a desiredpose) of a corresponding instrument located at a distal end of the linksand articulated joints of a corresponding articulated arm and/or endeffector. Each of the articulated joints proximal to the instrumentprovides the corresponding articulated arm and/or end effector with atleast one degree of freedom that may be used to manipulate the positionand/or orientation of the corresponding instrument. In many cases, thecorresponding articulated arms and/or end effectors may include at leastsix degrees of freedom that allow for controlling a x, y, and z position(collectively referred to as translational movement) of thecorresponding instrument as well as a roll, pitch, and yaw orientation(collectively referred to as rotational movement) of the correspondinginstrument. To provide for greater flexibility in control of the pose ofthe corresponding instrument, the corresponding articulated arms and/orend effectors are often designed to include redundant degrees offreedom. When redundant degrees of freedom are present it is possiblethat multiple different combinations of positions and/or orientations ofthe articulated joints may be used to obtain the same pose of thecorresponding instrument.

It is often desirable for the surgeon or operating room staff to move apatient on an operating or surgical table relative to the manipulatorarms of a computer-assisted device being used as a surgical manipulatorassembly in order to improve or optimize access to, or visualization of,the patient's internal anatomy. For example, a surgeon may wish toperform a gravity-assisted retraction of an organ during a surgicalprocedure. Because the patient's organs will move as the surgical tableis tilted, for safety the surgical instruments are removed from thepatient prior to moving the surgical table. Then, in many conventionalteleoperated surgical systems, to perform such a retraction, themanipulator arms must be undocked from the cannulas coupling the patientto the manipulator arms so that the body openings where the instrumentsare inserted into the patient can safely move, the surgical table mustthen be moved into a new position estimated to be suitable forretraction of the targeted organ, and then the instrument reinsertedinto the body openings. This method can be time consuming andcumbersome. Furthermore, this process may involve several iterations,because the endoscope is generally also removed from the patient beforethe table is moved to improve safety, such that visualization of thesurgical workspace is lost and the new position is typically an educatedguess, which may or may not be accurate or sufficient to properlyperform the retraction. To avoid repeated iterations, physicians often“overcorrect” and select positions and orientations that are steeperthan necessary to ensure that the desired gravity-assisted retractionoccurs. This overcorrection may lead to patient safety problems, becausecertain orientations, such as a head down orientation of the patient,may be poorly tolerated by a patient, and particularly by largerpatients who often have difficulty breathing in such an orientation. Inaddition, because the instruments are removed from the patient and themanipulator arms are removed from the cannulas, the instruments cannotbe used by a physician to assist with the retraction, such as may bedone in a traditional laparoscopic procedure.

Accordingly, it would be desirable to allow for repositioning of thepatient and or articulated arms while the articulated arms are connectedto a patient. It would also be desirable for a user to maintain somecontrol over the articulated arms as the patient or the arms are beingrepositioned. The systems and methods disclosed herein address theseproblems along with other problems.

SUMMARY

Consistent with some embodiments, a computer-assisted medical devicecomprises a first articulated arm, the first articulated arm having anend effector, a first joint set, and a second joint set; and a controlunit. In some embodiments, the control unit, when coupled to thearticulated arm and a surgical table, is configured configure one ormore joints in the first joint set to a floating mode; detect movementof the first joint set caused by a movement of the surgical table; drivethe second joint set based on the movement of the surgical table;receive an instrument motion command to move the end effector while thesurgical table is moving; and move the end effector based on theinstrument motion command. In some examples, the instrument motioncommand is relative to an imaging coordinate frame. In some examples thecontrol unit transforms the instrument motion command from the imagingcoordinate frame to a coordinate frame of the end effector. In someexamples the imaging coordinate frame is based on a pose of an imagingdevice saved prior to the movement of the surgical table. In someexamples the imaging coordinate frame is a virtual coordinate frame. Insome examples, the control unit maintains the imaging coordinate framein fixed relationship to a table top of the surgical table. In someexamples the control unit sends one or more movement commands to one ormore joints of the second joint set based on the instrument motioncommand. In some examples, the control unit is further configured todrive a third joint set in a second articulated arm based on themovement of the surgical table. In some examples the second joint set isdriven based on the movement of the surgical table and the control unitis configured to transform the movement of the surgical table in asurgical table coordinate frame into a motion in an end effectorcoordinate frame and drive the second joint set to move in relation tothe motion in the end effector coordinate frame.

In some embodiments, a method of controlling motion in a medical devicecomprises configuring one or more joints in a first joint set of a firstarticulated arm of the medical device to a floating mode, detectingmovement of the first joint set caused by movement of a surgical table,receiving data related to the movement of the surgical table, driving asecond joint set of the first articulated arm based on the receiveddata, receiving an instrument motion command to move an end effector ofthe first articulated arm while the surgical table is moving, and movingthe end effector based on the instrument motion command.

In some embodiments a non-transitory machine-readable medium comprises aplurality of machine-readable instructions which when executed by one ormore processors associated with a medical device are adapted to causethe one or more processors to perform a method comprising configuringone or more joints in a first joint set of a first articulated arm to afloating mode, detecting movement of the first joint set caused by asurgical table, receiving data related to the movement of the surgicaltable, driving a second joint set based on the data related to themovement of the surgical table, receiving an instrument motion commandto move the end effector while the surgical table is moving, and movingthe end effector based on the instrument motion command.

In some embodiments, a computer-assisted medical device comprises afirst articulated arm, the first articulated arm having an end effector.In some embodiments, a control unit, when coupled to the articulated armand a surgical table, is configured to maintain the orientation andposition of the end effector in relation to a table top of the surgicaltable when the surgical table is moved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified diagram of a computer-assisted system accordingto some embodiments.

FIG. 2 is a simplified diagram showing a computer-assisted systemaccording to some embodiments.

FIG. 3 is a simplified diagram of a kinematic model of acomputer-assisted medical system according to some embodiments.

FIG. 4A is a simplified diagram illustrating a perspective view of anexemplary camera view and coordinate systems.

FIG. 4B is a simplified diagram illustrating a camera view from theperspective of a sensor or a display and the related coordinate systems.

FIG. 5 is a simplified diagram of a method of maintaining control overone or more end effectors while one or more joints of an articulated armare moved when set to a floating state.

FIG. 6 is a simplified diagram of method for maintaining intuitivecontrols for an end effector on an articulated arm during table motion.

FIGS. 7A-7G are simplified schematic views that illustrate variouscomputer-assisted device system architectures that incorporate theintegrated computer-assisted device and movable surgical table featuresdescribed herein.

In the figures, elements having the same designations have the same orsimilar functions.

DETAILED DESCRIPTION

In the following description, specific details are set forth describingsome embodiments consistent with the present disclosure. It will beapparent to one skilled in the art, however, that some embodiments maybe practiced without some or all of these specific details. The specificembodiments disclosed herein are meant to be illustrative but notlimiting. One skilled in the art may realize other elements that,although not specifically described here, are within the scope and thespirit of this disclosure. In addition, to avoid unnecessary repetition,one or more features shown and described in association with oneembodiment may be incorporated into other embodiments unlessspecifically described otherwise or if the one or more features wouldmake an embodiment non-functional. The term “including” means includingbut not limited to, and each of the one or more individual itemsincluded should be considered optional unless otherwise stated.Similarly, the term “may” indicates that an item is optional.

FIG. 1 is a simplified diagram of a computer-assisted system 100according to some embodiments. As shown in FIG. 1 , computer-assistedsystem 100 includes a device 110 with one or more movable or articulatedarms 120. Each of the one or more articulated arms 120 supports one ormore end effectors. In some examples, device 110 may be consistent witha computer-assisted surgical device. The one or more articulated arms120 provides support for one or more instruments, surgical instruments,imaging devices, and/or the like mounted to a distal end of at least oneof the articulated arms 120. Device 110 may further be coupled to anoperator workstation (not shown), which may include one or more mastercontrols for operating the device 110, the one or more articulated arms120, and/or the end effectors. In some embodiments, device 110 and theoperator workstation may correspond to a da Vinci® Surgical Systemcommercialized by Intuitive Surgical, Inc. of Sunnyvale, Calif. In someembodiments, computer-assisted surgical devices with otherconfigurations, fewer or more articulated arms, and/or the like mayoptionally be used with computer-assisted system 100.

Device 110 is coupled to a control unit 130 via an interface. Theinterface may include one or more wireless links, cables, connectors,and/or buses and may further include one or more networks with one ormore network switching and/or routing devices. Control unit 130 includesa processor 140 coupled to memory 150. Operation of control unit 130 iscontrolled by processor 140. And although control unit 130 is shown withonly one processor 140, it is understood that processor 140 may berepresentative of one or more central processing units, multi-coreprocessors, microprocessors, microcontrollers, digital signalprocessors, field programmable gate arrays (FPGAs), application specificintegrated circuits (ASICs), and/or the like in control unit 130.Control unit 130 may be implemented as a stand-alone subsystem and/orboard added to a computing device or as a virtual machine. In someembodiments, control unit may be included as part of the operatorworkstation and/or operated separately from, but in coordination withthe operator workstation. Some examples of control units, such ascontrol unit 130 may include non-transient, tangible, machine readablemedia that include executable code that when run by one or moreprocessors (e.g., processor 140) may cause the one or more processors toperform the processes of methods 500 and 600.

Memory 150 is used to store software executed by control unit 130 and/orone or more data structures used during operation of control unit 130.Memory 150 may include one or more types of machine-readable media. Somecommon forms of machine readable media may include floppy disk, flexibledisk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, anyother optical medium, punch cards, paper tape, any other physical mediumwith patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memorychip or cartridge, and/or any other medium from which a processor orcomputer is adapted to read.

As shown, memory 150 includes a motion control application 160 thatsupports autonomous and/or semiautonomous control of device 110. Motioncontrol application 160 may include one or more application programminginterfaces (APIs) for receiving position, motion, and/or other sensorinformation from device 110, exchanging position, motion, and/orcollision avoidance information with other control units regarding otherdevices, such as a surgical table and/or imaging device, and/or planningand/or assisting in the planning of motion for device 110, articulatedarms 120, and/or the end effectors of device 110. And although motioncontrol application 160 is depicted as a software application, motioncontrol application 160 may be implemented using hardware, software,and/or a combination of hardware and software.

In some embodiments, computer-assisted system 100 may be found in anoperating room and/or an interventional suite. And althoughcomputer-assisted system 100 includes only one device 110 with twoarticulated arms 120, one of ordinary skill would understand thatcomputer-assisted system 100 may include any number of devices witharticulated arms and/or end effectors of similar and/or different designfrom device 110. In some examples, each of the devices may include feweror more articulated arms and/or end effectors.

Computer-assisted system 100 further includes a surgical table 170. Likethe one or more articulated arms 120, surgical table 170 supportsarticulated movement of a table top 180 relative to a base of surgicaltable 170. In some examples, the articulated movement of table top 180may include support for changing a height, a tilt, a slide, aTrendelenburg orientation, and/or the like of table top 180. Althoughnot shown, surgical table 170 may include one or more control inputs,such as a surgical table command unit for controlling the positionand/or orientation of table top 180. In some embodiments, surgical table170 may correspond to one or more of the surgical tables commercializedby Trumpf Medical Systems GmbH of Germany.

Surgical table 170 is also coupled to control unit 130 via acorresponding interface. The interface may include one or more wirelesslinks, cables, connectors, and/or buses and may further include one ormore networks with one or more network switching and/or routing devices.In some embodiments, surgical table 170 may be coupled to a differentcontrol unit than control unit 130. In some examples, motion controlapplication 160 may include one or more application programminginterfaces (APIs) for receiving position, motion, and/or other sensorinformation associated with surgical table 170 and/or table top 180. Insome examples, motion control application 160 may plan and/or assist inthe planning of motion for surgical table 170 and/or table top 180. Insome examples, motion control application 160 may contribute to motionplans associated with collision avoidance, adapting to and/or avoidrange of motion limits in joints and links, movement of articulatedarms, instruments, end effectors, surgical table components, and/or thelike to compensate for other motion in the articulated arms,instruments, end effectors, surgical table components, and/or the like,adjust a viewing device such as an endoscope to maintain and/or place anarea of interest and/or one or more instruments or end effectors withina field of view of the viewing device. In some examples, motion controlapplication 160 may prevent motion of surgical table 170 and/or tabletop 180, such as by preventing movement of surgical table 170 and/ortable top 180 through use of the surgical table command unit. In someexamples, motion control application 160 may help register device 110with surgical table 170 so that a geometric relationship between device110 and surgical table 170 is known. In some examples, the geometricrelationship may include a translation and/or one or more rotationsbetween coordinate frames maintained for device 110 and surgical table170.

Control unit 130 may further be coupled to an operator workstation 190via the interface. Operator workstation 190 may be used by an operator,such as a surgeon, to control the movement and/or operation of thearticulated arms 120 and the end effectors. To support operation of thearticulated arms 120 and the end effectors, operator workstation 190includes a display system 192 for displaying images of at least portionsof one or more of the articulated arms 120 and/or end effectors. Forexample, display system 192 may be used when it is impractical and/orimpossible for the operator to see the articulated arms 120 and/or theend effectors as they are being used. In some embodiments, displaysystem 192 may display a video image from a video capturing device, suchas an endoscope, which is controlled by one of the articulated arms 120,or a third articulated arm (not shown).

Operator workstation 190 may further include a console workspace withone or more input controls 195 (or “master controls 195”) that may beused for operating the device 110, the articulated arms 120, and/or theend effectors mounted on the articulated arms 120. Each of the inputcontrols 195 may be coupled to the distal end of their own articulatedarms so that movements of the input controls 195 may be detected by theoperator workstation 190 and communicated to control unit 130. Toprovide improved ergonomics, the console workspace may also include oneor more rests, such as an arm rest 197 on which operators may rest theirarms while manipulating the input controls 195. In some examples, thedisplay system 192 and the input controls 195 may be used by theoperator to teleoperate the articulated arms 120 and/or the endeffectors mounted on the articulated arms 120. In some embodiments,device 110, operator workstation 190, and control unit 130 maycorrespond to a da Vinci® Surgical System commercialized by IntuitiveSurgical, Inc. of Sunnyvale, Calif.

In some embodiments, other configurations and/or architectures may beused with computer-assisted system 100. In some examples, control unit130 may be included as part of operator workstation 190 and/or device110. In some embodiments, computer-assisted system 100 may be found inan operating room and/or an interventional suite. And althoughcomputer-assisted system 100 includes only one device 110 with twoarticulated arms 120, one of ordinary skill would understand thatcomputer-assisted system 100 may include any number of devices witharticulated arms and/or end effectors of similar and/or different designfrom device 110. In some examples, each of the devices may include feweror more articulated arms 120 and/or end effectors. Additionally, theremay be additional workstations 190 to control additional arms that maybe attached to device 110. Additionally, in some embodiments,workstation 190 may have controls for controlling surgical table 170.

FIG. 2 is a simplified diagram showing a computer-assisted system 200according to some embodiments. For example, the computer-assisted system200 may be consistent with computer-assisted system 100. As shown inFIG. 2 , the computer-assisted system 200 includes a computer-assisteddevice 210 with one or more articulated arms and a surgical table 280.Although not shown in FIG. 2 , the computer-assisted device 210 and thesurgical table 280 may be coupled together using one or more interfacesand one or more control units so that at least kinematic informationabout the surgical table 280 is known to the motion control applicationbeing used to perform motion of the articulated arms of thecomputer-assisted device 210.

The computer-assisted device 210 includes various links and joints. Inthe embodiments of FIG. 2 , the computer-assisted device is generallydivided into three different sets of links and joints. Starting at theproximal end with a mobile cart 215 (or “patient-side cart 215”) is aset-up structure 220. Coupled to a distal end of the set-up structure isa series of links and set-up joints 240 forming an articulated arm. Andcoupled to a distal end of the set-up joints 240 is a multi jointedmanipulator 260. In some examples, the series of set-up joints 240 andmanipulator 260 may correspond to one of the articulated arms 120. Andalthough the computer-assisted device is shown with only one series ofset-up joints 240 and a corresponding manipulator 260, one of ordinaryskill would understand that the computer-assisted device may includemore than one series of set-up joints 240 and corresponding manipulators260 so that the computer-assisted device is equipped with multiplearticulated arms.

As shown, the computer-assisted device 210 is mounted on the mobile cart215. The mobile cart 215 enables the computer-assisted device 210 to betransported from location to location, such as between operating roomsor within an operating room to better position the computer-assisteddevice in proximity to the surgical table 280. The set-up structure 220is mounted on the mobile cart 215. As shown in FIG. 2 , the set-upstructure 220 includes a two-part column including column links 221 and222. Coupled to the upper or distal end of the column link 222 is ashoulder joint 223. Coupled to the shoulder joint 223 is a two-part boomincluding boom links 224 and 225. At the distal end of the boom link 225is a wrist joint 226 and coupled to the wrist joint 226 is an armmounting platform 227.

The links and joints of the set-up structure 220 include various degreesof freedom for changing the position and orientation (i.e., the pose) ofthe arm mounting platform 227. For example, the two-part column is usedto adjust a height of the arm mounting platform 227 by moving theshoulder joint 223 up and down along an axis 232. The arm mountingplatform 227 is additionally rotated about the mobile cart 215, thetwo-part column, and the axis 232 using the shoulder joint 223. Thehorizontal position of the arm mounting platform 227 is also adjustedalong an axis 234 using the two-part boom. And the orientation of thearm mounting platform 227 may also adjusted by rotation about an armmounting platform orientation axis 236 using the wrist joint 226. Thus,subject to the motion limits of the links and joints in the set-upstructure 220, the position of the arm mounting platform 227 may beadjusted vertically above the mobile cart 215 using the two-part column.The positions of the arm mounting platform 227 may also be adjustedradially and angularly about the mobile cart 215 using the two-part boomand the shoulder joint 223, respectively. And the angular orientation ofthe arm mounting platform 227 may also be changed using the wrist joint226.

The arm mounting platform 227 is used as a mounting point for one ormore articulated arms. The ability to adjust the height, horizontalposition, and orientation of the arm mounting platform 227 about themobile cart 215 provides a flexible set-up structure for positioning andorienting the one or more articulated arms about a work space locatednear the mobile cart 215 where an operation or procedure is to takeplace. For example, arm mounting platform 227 may be positioned above apatient so that the various articulated arms and their correspondingmanipulators and instruments have sufficient range of motion to performa surgical procedure on the patient. FIG. 2 shows a single articulatedarm coupled to the arm mounting platform using a first set-up joint 242(or “flex joint 242.”) And although only one articulated arm is shown,one of ordinary skill would understand that multiple articulated armsmay be coupled to the arm mounting platform 227 using additional firstset-up joints.

The first set-up joint 242 forms the most proximal portion of the set-upjoints 240 section of the articulated arm that is the most proximal tothe patient-side cart 215. The set-up joints 240 may further include aseries of joints and links. As shown in FIG. 2 , the set-up joints 240may include at least links 244 and 246 coupled via one or more joints(not expressly shown). The joints and links of the set-up joints 240include the ability to rotate the set-up joints 240 relative to the armmounting platform 227 about an axis 252 using the first set-up joint242, adjust a radial or horizontal distance between the first set-upjoint 242 and the link 246, adjust a height of a manipulator mount 262at the distal end of link 246 relative to the arm mounting platform 227along an axis 254, and rotate the manipulator mount 262 about axis 254.In some examples, the set-up joints 240 may further include additionaljoints, links, and axes permitting additional degrees of freedom foraltering a pose of the manipulator mount 262 relative to the armmounting platform 227.

The manipulator 260 is coupled to the distal end of the set-up joints240 via the manipulator mount 262. The manipulator 260 includesadditional joints 264 and links 266 with an instrument carriage 268mounted at the distal end of the manipulator 260. An instrument ormanipulator instrument 270 is mounted to the instrument carriage 268.Instrument 270 includes a shaft 272, which is aligned along an insertionaxis. The shaft 272 is typically aligned so that it passes through aremote center of motion 274 associated with the manipulator 260.Location of the remote center of motion 274 is typically maintained in afixed translational relationship relative to the manipulator mount 262so that operation of the joints 264 in the manipulator 260 result inrotations of the shaft 272 about the remote center of motion 274.Depending upon the embodiment, the fixed translational relationship ofthe remote center of motion 274 relative to the manipulator mount 262 ismaintained using physical constraints in the joints 264 and links 266 ofthe manipulator 260, using software constraints placed on the motionspermitted for the joints 264, and/or a combination of both.Representative embodiments of computer-assisted surgical devices usingremote centers of motion maintained using physical constraints in jointsand links are described in U.S. patent application Ser. No. 13/906,888entitled “Redundant Axis and Degree of Freedom for Hardware-ConstrainedRemote Center Robotic Manipulator,” which was filed May 13, 2013, andrepresentative embodiments of computer-assisted surgical devices usingremote centers of motion maintained by software constraints aredescribed in U.S. Pat. No. 8,004,229 entitled “Software Center andHighly Configurable Robotic Systems for Surgery and Other Uses,” whichwas filed May 19, 2005, the specifications of which are herebyincorporated by reference in their entirety. In some examples, theremote center of motion 274 may correspond to a location of a surgicalport, body opening, or incision site in a patient 278 when shaft 272 isinserted into the patient 278. Because the remote center of motion 274corresponds to the body opening, as the instrument 270 is used, theremote center of motion 274 remains stationary relative to the patient278 to limit stresses on the anatomy of the patient 278 at the remotecenter of motion 274. In some examples, the shaft 272 may be optionallypassed through a cannula (not shown) located at the body opening. Insome examples, instruments having a relatively larger shaft or guidetube outer diameter (e.g., 4-5 mm or more) may be passed through thebody opening using a cannula and the cannula may optionally be omittedfor instruments having a relatively smaller shaft or guide tube outerdiameter (e.g., 2-3 mm or less).

At the distal end of the shaft 272 is an end effector 276. The degreesof freedom in the manipulator 260 due to the joints 264 and the links266 may permit at least control of the roll, pitch, and yaw of the shaft272 and/or end effector 276 relative to the manipulator mount 262. Insome examples, the degrees of freedom in the manipulator 260 may furtherinclude the ability to advance and/or withdraw the shaft 272 using theinstrument carriage 268 so that the end effector 276 may be advancedand/or withdrawn along the insertion axis and relative to the remotecenter of motion 274. In some examples, the manipulator 260 may beconsistent with a manipulator for use with the da Vinci® Surgical Systemcommercialized by Intuitive Surgical, Inc. of Sunnyvale, Calif. In someexamples, the instrument 270 may be an imaging device such as anendoscope, a gripper, a surgical instrument such as a cautery or ascalpel, and/or the like. In some examples, the end effector 276 mayinclude additional degrees of freedom, such as roll, pitch, yaw, grip,and/or the like that allow for additional localized manipulation ofportions of the end effector 276 relative to the distal end of shaft272.

During a surgery or other medical procedure, the patient 278 istypically located on the surgical table 280. The surgical table 280includes a table base 282 and a table top 284 with the table base 282being located in proximity to mobile cart 215 so that the instrument 270and/or end effector 276 may be manipulated by the computer-assisteddevice 210 while the shaft 272 of instrument 270 is inserted into thepatient 278 at the body opening. The surgical table 280 further includesan articulated structure 290 that includes one or more joints or linksbetween the table base 282 and the table top 284 so that the relativelocation of the table top 284, and thus the patient 278, relative to thetable base 282 is controlled. In some examples, the articulatedstructure 290 may be configured so that the table top 284 is controlledrelative to a virtually-defined table motion isocenter 286 that may belocated at a point above the table top 284. In some examples, isocenter286 may be located within the interior of the patient 278. In someexamples, isocenter 286 may be collocated with the body wall of thepatient at or near one of the body openings, such as a body openingcorresponding to remote center of motion 274.

As shown in FIG. 2 , the articulated structure 290 includes a heightadjustment joint 292 so that the table top 284 may be raised and/orlowered relative to the table base 282. The articulated structure 290further includes joints and links to change both the tilt 294 andTrendelenburg 296 orientation of the table top 284 relative to theisocenter 286. The tilt 294 allows the table top 284 to be tiltedside-to-side so that either the right or left side of the patient 278may be rotated upward relative to the other side of the patient 278(i.e., about a longitudinal, cranial-caudal, or head-to-toe(cranial-caudal) axis of the table top 284). The Trendelenburg 296allows the table top 284 to be rotated so that either the feet of thepatient 278 are raised (Trendelenburg) or the head of the patient 278 israised (reverse Trendelenburg). In some examples, either the tilt 294and/or the Trendelenburg 296 rotations may be adjusted to generaterotations about isocenter 286. The articulated structure 290 furtherincludes additional links and joints 298 to slide the table top 284along the longitudinal (cranial-caudal) axis back and forth relative tothe table base 282 with generally a left and/or right motion as depictedin FIG. 2 .

FIGS. 7A-7G are simplified schematic views that illustrate variouscomputer-assisted device system architectures that incorporate theintegrated computer-assisted device and movable surgical table featuresdescribed herein. The various illustrated system components are inaccordance with the principles described herein. In these illustrations,the components are simplified for clarity, and various details such asindividual links, joints, manipulators, instruments, end effectors, etc.are not shown, but they should be understood to be incorporated in thevarious illustrated components.

In these architectures, cannulas associated with one or more surgicalinstruments or clusters of instruments are not shown, and it should beunderstood that cannulas and other instrument guide devices optionallymay be used for instruments or instrument clusters having a relativelylarger shaft or guide tube outer diameter (e.g., 4-5 mm or more) andoptionally may be omitted for instruments having a relatively smallershaft or guide tube outer diameter (e.g., 2-3 mm or less).

Also in these architectures, teleoperated manipulators should beunderstood to include manipulators that during surgery define a remotecenter of motion by using hardware constraints (e.g., fixed intersectinginstrument pitch, yaw, and roll axes) or software constraints (e.g.,software-constrained intersecting instrument pitch, yaw, and roll axes).A hybrid of such instrument axes of rotation may be defined (e.g.,hardware-constrained roll axis and software-constrained pitch and yawaxes) are also possible. Further, some manipulators may not define andconstrain any surgical instrument axes of rotation during a procedure,and some manipulators may define and constrain only one or twoinstrument axes of rotation during a procedure.

FIG. 7A illustrates a movable surgical table 1100 and asingle-instrument computer-assisted device 1101 a are shown. Surgicaltable 1100 includes a movable table top 1102 and a table supportstructure 1103 that extends from a mechanically grounded table base 1104to support the table top 1102 at a distal end. In some examples,surgical table 1100 may be consistent with surgical table 170 and/or280. Computer-assisted device 1101 a includes a teleoperated manipulatorand a single instrument assembly 1105 a. Computer-assisted device 1101 aalso includes a support structure 1106 a that is mechanically groundedat a proximal base 1107 a and that extends to support manipulator andinstrument assembly 1105 a at a distal end. Support structure 1106 a isconfigured to allow assembly 1105 a to be moved and held in variousfixed poses with reference to surgical table 1100. Base 1107 a isoptionally permanently fixed or movable with reference to surgical table1100. Surgical table 1100 and computer-assisted device 1101 a operatetogether as described herein.

FIG. 7A further shows an optional second computer-assisted device 1101b, which illustrates that two, three, four, five, or more individualcomputer-assisted devices may be included, each having a correspondingindividual teleoperated manipulator and single-instrument assembly(ies)1105 b supported by a corresponding support structure 1106 b.Computer-assisted device 1101 b is mechanically grounded, and assemblies1105 b are posed, similarly to computer-assisted device 1101 a. Surgicaltable 1100 and computer-assisted devices 1101 a and 1101 b together makea multi-instrument surgical system, and they operate together asdescribed herein. In some examples, computer-assisted devices 1101 aand/or 1101 b may be consistent with computer-assisted devices 110and/or 210.

As shown in FIG. 7B, another movable surgical table 1100 and acomputer-assisted device 1111 are shown. Computer-assisted device 1111is a multi-instrument device that includes two, three, four, five, ormore individual teleoperated manipulator and single-instrumentassemblies as shown by representative manipulator and instrumentassemblies 1105 a and 1105 b. The assemblies 1105 a and 1105 b ofcomputer-assisted device 1111 are supported by a combined supportstructure 1112, which allows assemblies 1105 a and 1105 b to be movedand posed together as a group with reference to surgical table 1100. Theassemblies 1105 a and 1105 b of computer-assisted device 1111 are alsoeach supported by a corresponding individual support structure 1113 aand 1113 b, respectively, which allows each assembly 1105 a and 1105 bto be individually moved and posed with reference to surgical table 1100and to the one or more other assemblies 1105 a and 1105 b. Examples ofsuch a multi-instrument surgical system architecture are the da VinciSi® Surgical System and the da Vinci® Xi™ Surgical System,commercialized by Intuitive Surgical, Inc. Surgical table 1100 and asurgical manipulator system comprising an example computer-assisteddevice 1111 operate together as described herein. In some examples,computer-assisted device 1111 is consistent with computer-assisteddevices 110 and/or 210.

The computer-assisted devices of FIGS. 7A and 7B are each shownmechanically grounded at the floor. But, one or more suchcomputer-assisted devices may optionally be mechanically grounded at awall or ceiling and be permanently fixed or movable with reference tosuch a wall or ceiling ground. In some examples, computer-assisteddevices may be mounted to the wall or ceiling using a track or gridsystem that allows the support base of the computer-assisted systems tobe moved relative to the surgical table. In some examples, one or morefixed or releasable mounting clamps may be used to mount the respectivesupport bases to the track or grid system. As shown in FIG. 7C, acomputer-assisted device 1121 a is mechanically grounded at a wall, anda computer-assisted device 1121 b is mechanically grounded at a ceiling.

In addition, computer-assisted devices may be indirectly mechanicallygrounded via the movable surgical table 1100. As shown in FIG. 7D, acomputer-assisted device 1131 a is coupled to the table top 1102 ofsurgical table 1100. Computer-assisted device 1131 a may optionally becoupled to other portions of surgical table 1100, such as table supportstructure 1103 or table base 1104, as indicated by the dashed structuresshown in FIG. 7D. When table top 1102 moves with reference to tablesupport structure 1103 or table base 1104, the computer-assisted device1131 a likewise moves with reference to table support structure 1103 ortable base 1104. When computer-assisted device 1131 a is coupled totable support structure 1103 or table base 1104, however, the base ofcomputer-assisted device 1131 a remains fixed with reference to groundas table top 1102 moves. As table motion occurs, the body opening whereinstruments are inserted into the patient may move as well because thepatient's body may move and change the body opening locations relativeto the table top 1102. Therefore, for embodiments in whichcomputer-assisted device 1131 a is coupled to the table top 1102, thetable top 1102 functions as a local mechanical ground, and the bodyopenings move with reference to the table top 1102, and so withreference to the computer-assisted device 1131 a as well. FIG. 7D alsoshows that a second computer-assisted device 1131 b optionally may beadded, configured similarly to computer-assisted device 1131 a to createa multi-instrument system. Systems that include one or morecomputer-assisted device coupled to the surgical table operate asdisclosed herein.

In some embodiments, other combinations of computer-assisted deviceswith the same or hybrid mechanical groundings are possible. For example,a system may include one computer-assisted device mechanically groundedat the floor, and a second computer-assisted device mechanicallygrounded to the floor via the surgical table. Such hybrid mechanicalground systems operate as disclosed herein.

Inventive aspects also include single-body opening systems in which twoor more surgical instruments enter the body via a single body opening.Examples of such systems are shown in U.S. Pat. No. 8,852,208 entitled“Surgical System Instrument Mounting,” which was filed Aug. 12, 2010,and U.S. Pat. No. 9,060,678 entitled “Minimally Invasive SurgicalSystem,” which was filed Jun. 13, 2007, both of which are incorporatedby reference. FIG. 7E illustrates a teleoperated multi-instrumentcomputer-assisted device 1141 together with surgical table 1100 asdescribed above. Two or more instruments 1142 are each coupled to acorresponding manipulator 1143, and the cluster of instruments 1142 andinstrument manipulators 1143 are moved together by a system manipulator1144. The system manipulator 1144 is supported by a support assembly1145 that allows system manipulator 1144 to be moved to and fixed atvarious poses. Support assembly 1145 is mechanically grounded at a base1146 consistent with the descriptions above. The two or more instruments1142 are inserted into the patient at the single body opening.Optionally, the instruments 1142 extend together through a single guidetube, and the guide tube optionally extends through a cannula, asdescribed in the references cited above. Computer-assisted device 1141and surgical table 1100 operate together as described herein.

FIG. 7F illustrates another multi-instrument, single-body openingcomputer-assisted device 1151 mechanically grounded via the surgicaltable 1100, optionally by being coupled to table top 1102, table supportstructure 1103, or table base 1104. The descriptions above withreference to FIG. 7D also applies to the mechanical grounding optionsillustrated in FIG. 7F. Computer-assisted device 1151 and surgical table1100 work together as described herein.

FIG. 7G illustrates that one or more teleoperated multi-instrument,single-body opening computer-assisted devices 1161 and one or moreteleoperated single-instrument computer-assisted devices 1162 may becombined to operate with surgical table 1100 as described herein. Eachof the computer-assisted devices 1161 and 1162 may be mechanicallygrounded, directly or via another structure, in various ways asdescribed above.

FIG. 3 is a simplified diagram of a kinematic model 300 of acomputer-assisted medical system according to some embodiments. As shownin FIG. 3 , kinematic model 300 may include kinematic informationassociated with many sources and/or devices. The kinematic informationmay be based on known kinematic models for the links and joints of acomputer-assisted medical device and a surgical table. The kinematicinformation may be further based on information associated with theposition and/or orientation of the joints of the computer-assistedmedical device and the surgical table. In some examples, the informationassociated with the position and/or orientation of the joints may bederived from one or more sensors, such as encoders, measuring the linearpositions of prismatic joints and the rotational positions of revolutejoints.

The kinematic model 300 includes several coordinate frames or coordinatesystems and transformations, such as homogeneous transforms, fortransforming positions and/or orientation from one of the coordinateframes to another of the coordinate frames. In some examples, thekinematic model 300 may be used to permit the forward and/or reversemapping of positions and/or orientations in one of the coordinate framesin any other of the coordinate frames by composing the forward and/orreverse/inverse transforms noted by the transform linkages included inFIG. 3 . In some examples, when the transforms are modeled as homogenoustransforms in matrix form, the composing may be accomplished usingmatrix multiplication. In some embodiments, a system may use theDenavit-Hartenberg parameters and conventions for attaching coordinatereference frames to one or more points in the kinematic chain andtransforming from one reference frame to the other in the kinematicmodel 300. In some embodiments, the kinematic model 300 may be used tomodel the kinematic relationships of the computer-assisted device 210and the surgical table 280 of FIG. 2 .

The kinematic model 300 includes a table base coordinate frame 305 thatmay be used to model a position and/or orientation of a surgical table,such as surgical table 170 and/or surgical table 280. In some examples,the table base coordinate frame 305 may be used to model other points onthe surgical table relative to a reference point and/or orientationassociated with the surgical table. In some examples, the referencepoint and/or orientation may be associated with a table base of thesurgical table, such as the table base 282. In some examples, the tablebase coordinate frame 305 may be suitable for use as a world coordinateframe for the computer-assisted system.

The kinematic model 300 further includes a table top coordinate frame310 that may be used to model positions and/or orientations in acoordinate frame representative of a table top of the surgical table,such as the table top 284. In some examples, the table top coordinateframe 310 may be centered about a rotational center or isocenter of thetable top, such as isocenter 286. In some examples, the z-axis of thetable top coordinate frame 310 may be oriented vertically with respectto a floor or surface on which the surgical table is placed and/ororthogonal to the surface of the table top. In some examples, the x- andy-axes of the table top coordinate frame 310 may be oriented to capturethe longitudinal (head to toe) and lateral (side-to-side) major axes ofthe table top. In some examples, a table base to table top coordinatetransform 315 is used to map positions and/or orientations between thetable top coordinate frame 310 and the table base coordinate frame 305.In some examples, one or more kinematic models of an articulatedstructure of the surgical table, such as articulated structure 290,along with past and/or current joint sensor readings is used todetermine the table base to table top coordinate transform 315. In someexamples consistent with the embodiments of FIG. 2 , the table base totable top coordinate transform 315 models the composite effect of theheight, tilt, Trendelenburg, and/or slide settings associated with thesurgical table.

The kinematic model 300 further includes a device base coordinate framethat is used to model a position and/or orientation of acomputer-assisted device, such as computer-assisted device 110 and/orcomputer-assisted device 210. In some examples, the device basecoordinate frame 320 may be used to model other points on thecomputer-assisted device relative to a reference point and/ororientation associated with the computer-assisted device. In someexamples, the reference point and/or orientation may be associated witha device base of the computer-assisted device, such as the mobile cart215. In some examples, the device base coordinate frame 320 may besuitable for use as the world coordinate frame for the computer-assistedsystem.

In order to track positional and/or orientational relationships betweenthe surgical table and the computer-assisted device, it is oftendesirable to perform a registration between the surgical table and thecomputer-assisted device. As shown in FIG. 3 , the registration may beused to determine a registration transform 325 between the table topcoordinate frame 310 and the device base coordinate from 320. In someembodiments, the registration transform 325 may be a partial or fulltransform between the table top coordinate frame 310 and the device basecoordinate frame 320. The registration transform 325 is determined basedon the architectural arrangements between the surgical table and thecomputer-assisted device.

In the examples of FIGS. 7D and 7F, where the computer-assisted deviceis mounted to the table top 1102, the registration transform 325 isdetermined from the table base to table top coordinate transform 315 andknowing where the computer-assisted device is mounted to the table top112.

In the examples of FIGS. 7A-7C, 7E, and 7F, where the computer-assisteddevice is placed on the floor or mounted to the wall or ceiling,determination of the registration transform 325 is simplified by placingsome restrictions on the device base coordinate frame 320 and the tablebase coordinate frame 305. In some examples, these restrictions includethat both the device base coordinate frame 320 and the table basecoordinate frame 305 agree on the same vertical up or z-axis. Under theassumption that the surgical table is located on a level floor, therelative orientations of the walls of the room (e.g., perpendicular tothe floor) and the ceiling (e.g., parallel to the floor) are known it ispossible for a common vertical up or z axis (or a suitable orientationtransform) to be maintained for both the device base coordinate frame320 and the table base coordinate frame 305 or a suitable orientationtransform. In some examples, because of the common z-axis, theregistration transform 325 may optionally model just the rotationalrelationship of the device base to the table base about the z-axis ofthe table base coordinate frame 305 (e.g., a θz registration). In someexamples, the registration transform 325 may optionally also model ahorizontal offset between the table base coordinate frame 305 and thedevice base coordinate frame 320 (e.g., a XY registration). This ispossible because the vertical (z) relationship between thecomputer-assisted device and the surgical table are known. Thus, changesin a height of the table top in the table base to table top transform315 are analogous to vertical adjustments in the device base coordinateframe 320 because the vertical axes in the table base coordinate frame305 and the device base coordinate frame 320 are the same or nearly thesame so that changes in height between the table base coordinate frame305 and the device base coordinate frame 320 are within a reasonabletolerance of each other. In some examples, the tilt and Trendelenburgadjustments in the table base to table top transform 315 may be mappedto the device base coordinate frame 320 by knowing the height of thetable top (or its isocenter) and the θz and/or XY registration. In someexamples, the registration transform 325 and the table base to table toptransform 315 may be used to model the computer-assisted surgical deviceas if it were attached to the table top even when this isarchitecturally not the case.

The kinematic model 300 further includes an arm mounting platformcoordinate frame 330 that is used as a suitable model for a sharedcoordinate frame associated with the most proximal points on thearticulated arms of the computer-assisted device. In some embodiments,the arm mounting platform coordinate frame 330 may be associated withand oriented relative to a convenient point on an arm mounting platform,such as the arm mounting platform 227. In some examples, the centerpoint of the arm mounting platform coordinate frame 330 may be locatedon the arm mounting platform orientation axis 236 with the z-axis of thearm mounting platform coordinate frame 330 being aligned with armmounting platform orientation axis 236. In some examples, a device baseto arm mounting platform coordinate transform 335 may be used to mappositions and/or orientations between the device base coordinate frame320 and the arm mounting platform coordinate frame 330. In someexamples, one or more kinematic models of the links and joints of thecomputer-assisted device between the device base and the arm mountingplatform, such as the set-up structure 220, along with past and/orcurrent joint sensor readings may be used to determine the device baseto arm mounting platform coordinate transform 335. In some examplesconsistent with the embodiments of FIG. 2 , the device base to armmounting platform coordinate transform 335 may model the compositeeffect of the two-part column, shoulder joint, two-part boom, and wristjoint of the setup structure portion of the computer-assisted device.

The kinematic model 300 further includes a series of coordinate framesand transforms associated with each of the articulated arms of thecomputer-assisted device. As shown in FIG. 3 , the kinematic model 300includes coordinate frames and transforms for three articulated arms,although one of ordinary skill would understand that differentcomputer-assisted devices may include fewer and/or more articulated arms(e.g., one, two, four, five, or more). Consistent with the configurationof the links and joints of the computer-assisted device 210 of FIG. 2 ,each of the articulated arms may be modeled using a manipulator mountcoordinate frame, a remote center of motion coordinate frame, and aninstrument or camera coordinate frame, depending on a type of instrumentmounted to the distal end of the articulated arm.

In the kinematic model 300, the kinematic relationships of a first oneof the articulated arms is captured using a manipulator mount coordinateframe 341, a remote center of motion coordinate frame 342, an instrumentcoordinate frame 343, an arm mounting platform to manipulator mounttransform 344, a manipulator mount to remote center of motion transform345, and a remote center of motion to instrument transform 346. Themanipulator mount coordinate frame 341 represents a suitable model forrepresenting positions and/or orientations associated with amanipulator, such as manipulator 260. The manipulator mount coordinateframe 341 is typically associated with a manipulator mount, such as themanipulator mount 262 of the corresponding articulated arm. The armmounting platform to manipulator mount transform 344 is then based onone or more kinematic models of the links and joints of thecomputer-assisted device between the arm mounting platform and thecorresponding manipulator mount, such as the corresponding set-up joints240, along with past and/or current joint sensor readings of thecorresponding set-up joints 240.

The remote center of motion coordinate frame 342 is typically associatedwith a remote center of motion of the instrument mounted on themanipulator, such as the corresponding remote center of motion 274 ofthe corresponding manipulator 260. The manipulator mount to remotecenter of motion transform 345 is then based on one or more kinematicmodels of the links and joints of the computer-assisted device betweenthe corresponding manipulator mount and the corresponding remote centerof motion, such as the corresponding joints 264, corresponding links266, and corresponding carriage 268 of the corresponding manipulator260, along with past and/or current joint sensor readings of thecorresponding joints 264. When the corresponding remote center of motionis being maintained in fixed positional relationship to thecorresponding manipulator mounts, such as in the embodiments of FIG. 2 ,the manipulator mount to remote center of motion transform 345 includesan essentially static translational component that does not change asthe manipulator and instrument are operated and a dynamic rotationalcomponent that changes as the manipulator and instrument are operated.

The instrument coordinate frame 343 is typically associated with a pointon an instrument and/or end effector on an instrument mounted on thearticulated arm, such as the corresponding end effector 276 oncorresponding instrument 270. The remote center of motion to instrumenttransform 346 is then based on one or more kinematic models of the linksand joints of the computer-assisted device that move and/or orient thecorresponding instrument, end effector, and the corresponding remotecenter of motion, along with past and/or current joint sensor readings.In some examples, the remote center of motion to instrument transform346 accounts for the orientation at which the shaft, such as thecorresponding shaft 272, passes through the remote center of motion andthe distance to which the shaft is advanced and/or withdrawn relative tothe remote center of motion. In some examples, the remote center ofmotion to instrument transform 346 may further be constrained to reflectthat the insertion axis of the shaft of the instrument passes throughthe remote center of motion and may account for rotations of the shaftand the end effector about the axis defined by the shaft.

In the kinematic model 300, the kinematic relationships of a second oneof the articulated arms is captured using a manipulator mount coordinateframe 351, a remote center of motion coordinate frame 352, an instrumentcoordinate frame 353 (or “instrument reference frame 353”), an armmounting platform to manipulator mount transform 354, a manipulatormount to remote center of motion transform 355, and a remote center ofmotion to instrument transform 356. The manipulator mount coordinateframe 351 represents a suitable model for representing positions and/ororientations associated with a manipulator, such as manipulator 260. Themanipulator mount coordinate frame 351 is typically associated with amanipulator mount, such as the manipulator mount 262 of thecorresponding articulated arm. The arm mounting platform to manipulatormount transform 354 is then based on one or more kinematic models of thelinks and joints of the computer-assisted device between the armmounting platform and the corresponding manipulator mount, such as thecorresponding set-up joints 240, along with past and/or current jointsensor readings of the corresponding set-up joints 240.

The remote center of motion coordinate frame 352 is associated with aremote center of motion of the manipulator mounted on the articulatedarm, such as the corresponding remote center of motion 274 of thecorresponding manipulator 260. The manipulator mount to remote center ofmotion transform 355 is then based on one or more kinematic models ofthe links and joints of the computer-assisted device between thecorresponding manipulator mount and the corresponding remote center ofmotion, such as the corresponding joints 264, corresponding links 266,and corresponding carriage 268 of the corresponding manipulator 260,along with past and/or current joint sensor readings of thecorresponding joints 264. When the corresponding remote center of motionis being maintained in fixed positional relationship to thecorresponding manipulator mounts, such as in the embodiments of FIG. 2 ,the manipulator mount to remote center of motion transform 355 mayinclude an essentially static translational component that does notchange as the manipulator and instrument are operated and a dynamicrotational component that changes as the manipulator and instrument areoperated.

The instrument coordinate frame 353 is associated with a point on an endeffector, instrument, and/or end effector on an instrument mounted onthe articulated arm, such as the corresponding end effector 276 oncorresponding instrument 270 and/or end effector 276. The remote centerof motion to instrument transform 356 is then based on one or morekinematic models of the links and joints of the computer-assisted devicethat move and/or orient the corresponding instrument, end effector, andremote center of motion, along with past and/or current joint sensorreadings. In some examples, the remote center of motion to instrumenttransform 356 accounts for the orientation at which the shaft, such asthe corresponding shaft 272, passes through the remote center of motionand the distance to which the shaft is advanced and/or withdrawnrelative to the remote center of motion. In some examples, the remotecenter of motion to instrument transform 356 may be constrained toreflect that the insertion axis of the shaft of the instrument passesthrough the remote center of motion and accounts for rotations of theshaft and the end effector about the insertion axis defined by theshaft.

In the kinematic model 300, the kinematic relationships of a third oneof the articulated arms is captured using a manipulator mount coordinateframe 361, a remote center of motion coordinate frame 362, a cameracoordinate frame 363, an arm mounting platform to manipulator mounttransform 364, a manipulator mount to remote center of motion transform365, and a remote center of motion to camera transform 366. Themanipulator mount coordinate frame 361 represents a suitable model forrepresenting positions and/or orientations associated with amanipulator, such as manipulator 260. The manipulator mount coordinateframe 361 is typically associated with a manipulator mount, such as themanipulator mount 262 of the corresponding articulated arm. The armmounting platform to manipulator mount transform 364 is then based onone or more kinematic models of the links and joints of thecomputer-assisted device between the arm mounting platform and thecorresponding manipulator mount, such as the corresponding set-up joints240, along with past and/or current joint sensor readings of thecorresponding set-up joints 240.

The remote center of motion coordinate frame 362 is typically associatedwith a remote center of motion of the manipulator of the articulatedarm, such as the corresponding remote center of motion 274 of thecorresponding manipulator 260. The manipulator mount to remote center ofmotion transform 365 is then based on one or more kinematic models ofthe links and joints of the computer-assisted device between thecorresponding manipulator mount and the corresponding remote center ofmotion, such as the corresponding joints 264, corresponding links 266,and corresponding carriage 268 of the corresponding manipulator 260,along with past and/or current joint sensor readings of thecorresponding joints 264. When the corresponding remote center of motionis being maintained in fixed positional relationship to thecorresponding manipulator mounts, such as in the embodiments of FIG. 2 ,the manipulator mount to remote center of motion transform 365 mayinclude an essentially static translational component that does notchange as the manipulator and instrument are operated and a dynamicrotational component that changes as the manipulator and instrument areoperated.

The camera coordinate frame 363 is associated with an imaging device,such an endoscope, mounted on the articulated arm. The remote center ofmotion to camera transform 366 is then based on one or more kinematicmodels of the links and joints of the computer-assisted device that moveand/or orient the imaging device and the corresponding remote center ofmotion, along with past and/or current joint sensor readings. In someexamples, the remote center of motion to camera transform 366 accountsfor the orientation at which the shaft, such as the corresponding shaft272, passes through the remote center of motion and the distance towhich the shaft is advanced and/or withdrawn relative to the remotecenter of motion. In some examples, the remote center of motion tocamera transform 366 may be constrained to reflect that the insertionaxis of the shaft of the imaging device passes through the remote centerof motion and accounts for rotations of the imaging device about theaxis defined by the shaft.

In some embodiments, an imaging device associated with camera coordinateframe 363 may stream video to an operator workstation such that a usermay view the video stream from camera coordinate frame 363. For example,the video captured by the imaging device may be relayed and displayed ondisplay system 192 of operator workstation 190 of FIG. 1 . In someembodiments, the imaging device may be oriented such that it capturesvideo and/or images of an instrument associated with instrumentcoordinate frame 343 and/or an instrument associated with instrumentcoordinate frame 353. The instrument associated with instrumentcoordinate frame 343 and/or the instrument associated with instrumentcoordinate frame 353 may be operated by the user through a controller,such as input or master controls 195 of FIG. 1 . In some embodiments, toallow for intuitive manipulation of the instruments, user commands fromthe controls may correlate with the coordinate system of the cameracoordinate frame 363. For example, commands of up and down, left andright, and in and out using the controllers may translate to movementsof the instruments up and down, left and right, and in and out inrelation to camera coordinate frame 363. Up and down, left and right, inand out, may be resented by the x, y, and z translational axis ofcoordinate frame 363. Similarly, roll, pitch, and yaw commands may causethe instrument to roll, pitch, and yaw in relation to the cameracoordinate frame. In some embodiments, one or more processors, such asprocessor 140 of FIG. 1 , may translate user commands from the cameracoordinate frame 363 to respective commands and motion in the instrumentcoordinate frames 343 and 353. The translational commands may be throughthe kinematic relationships. For example, commands to the instrumentassociated with instrument coordinate frame 343 may go from cameracoordinate frame 363 to remote center of motion reference frame 362using transform 366, then from remote center of motion reference frame362 to manipulator mount coordinate frame 361 using transform 365,manipulator mount coordinate frame 361 to arm mounting platformcoordinate frame 330 using transform 364, arm mounting platformcoordinate frame 330 to manipulator mount coordinate frame 341 usingtransform 344, manipulator mount coordinate frame 341 to remote centerof motion coordinate frame 342 using transform 345, and remote center ofmotion coordinate frame 342 to instrument coordinate frame 343 usingtransform 346. In this manner, any motion commands known in onereference frame can be transformed to corresponding commands in one ormore other coordinate frames.

As discussed above and further emphasized here, FIG. 3 is merely anexample which should not unduly limit the scope of the claims. One ofordinary skill in the art would recognize many variations, alternatives,and modifications. According to some embodiments, the registrationbetween the surgical table and the computer-assisted device may bedetermined between the table top coordinate frame 310 and the devicebase coordinate frame 320 using an alternative registration transform.When the alternative registration transform is used, registrationtransform 325 is determined by composing the alternative registrationtransform with the inverse/reverse of the table base to table toptransform 315. According to some embodiments, the coordinate framesand/or transforms used to model the computer-assisted device may bearranged differently dependent on the particular configuration of thelinks and joints of the computer-assisted device, its articulated arms,its end effectors, its manipulators, and/or its instruments. Accordingto some embodiments, the coordinate frames and transforms of thekinematic model 300 may be used to model coordinate frames andtransforms associated with one or more virtual instruments and/orvirtual cameras. In some examples, the virtual instruments and/orcameras may be associated with previously stored and/or latchedinstrument positions, projections of instruments and/or cameras due to amotion, reference points defined by a surgeon and/or other personnel,and/or the like.

As a computer-assisted device, such as computer-assisted devices 110and/or 210, is being operated, one of the goals is to minimize and/oreliminate the propagation of disturbances and/or movements from one ormore joints and/or links of an articulated arm to the position of one ormore points of an instrument, link(s), and or joint(s). For example,referring to FIG. 2 , a disturbance to one or more of joints 242 and/orlinks 246 may cause an injury to patient 278 if the disturbance ispropagated to end effector 276 (end effector 276 being an exemplarypoint of interest) while inside of patient 278.

In one mode of operation for the computer-assisted system, one or morejoints of the surgical table and joints of the articulated arms may belocked and/or held in place through the use of servo control and/orbrakes so that motion of the joints is limited and/or prohibitedentirely. In some examples, this allows the joints of the manipulatorsto control an instrument undisturbed by motion from other joints whenaccomplishing a desired procedure. In some embodiments, the manipulatorsmay be physically constrained to maintain a remote center of motion andmotion of one or more joints that do not make up the manipulator mightundesirably cause the remote center of motion to move. In thoseexamples, it may be beneficial to have the joints that do not make upthe manipulators be locked in place through physical and/or servocontrol braking systems. However, there may be instances where allowingmovement of the remote center of motion would be desirable, and thusallowing for release of the brakes locking one or more of the jointsthat may affect the position of the remote center of motion.

In some examples, the instruments may be inserted into a patient duringthe procedure. In some examples, the position of the instruments may becontrolled via teleoperation by a surgeon at an operator console such asworkstation 190 of FIG. 1 . It may, however, be desirable to supportother modes of operation for the computer-assisted system that allow formovement in the articulated arms while the instruments remain insertedinto the patient. These other modes of operation may increase the risksrelative to modes of operation when the instruments are not insertedinto to the patient. In some examples, these risks may include injury tothe patient when the instruments are allowed to move relative to thepatient, breach of a sterile field, collisions between the articulatedarms, and/or the like.

In a general case, these other modes of operation may be characterizedby a goal of maintaining a point of an instrument relative to a patientwhen one or more joints proximal to the instrument are subject to adisturbance that results in a change to positions and/or orientations(i.e., movements) of the one or more joints. Because disturbances in oneor more first joints, which may be referred to as disturbed joints,proximal to an instrument results in a change in the position of theinstrument, it may be desirable to introduce movement in one or moresecond or compensating joints that compensate for the movement of theinstrument caused by the movement of the disturbed joints. Determiningthe extent of the disturbance and the amount of compensation depends onthe type and nature of the disturbance, such as whether the disturbanceis associated with movement of the surgical table or patient, or whetherthe disturbance is confined to the articulated arm used to control theinstrument.

The disturbances associated with these other modes of operation thatmaintain a position of an instrument may occur when the patient ismoving so that the position of the instrument and/or the end effectormay be monitored in a local coordinate frame. In some examples, thesedisturbances may include disturbances caused by allowing motion of thearticulated structure in the surgical table (i.e., table movement) ormovement of the patient relative to the surgical table. In someexamples, it is generally desired to have the articulated arm and theinstrument move with the patient so that the position of the instrumentrelative to the patient does not change. In some examples, this may beaccomplished using instrument dragging that may include releasing and/orunlocking one or more joints of the articulated arm and allowing thebody wall of the patient at the body opening to drag the remote centerof motion and the instrument as the patient moves. In some examples, asthe remote center of motion moves the orientation of the instrumentrelative to the remote center of motion may begin to change resulting ina change between the position of instrument relative to the patient.Examples of systems permitting active continuation of a surgicalprocedure during surgical table motion are shown in U.S. ProvisionalPatent Application No. 62/134,207 entitled “System and Method forIntegrated Surgical Table,” which was filed Mar. 17, 2015, andconcurrently filed PCT Patent Application No. PCT/US2015/057656 entitled“System and Method for Integrated Surgical Table” and published asWO2016/069648 A1, both of which are hereby incorporated by reference intheir entirety.

FIGS. 4A and 4B illustrate an exemplary camera view 400 from twodifferent perspectives. FIG. 4A may be an overhead perspective, and FIG.4B may be the perspective of a sensor of imaging device 410. Camera view400 from the perspective of FIG. 4B may be viewed from a display, suchas display system 192 of operator workstation 190 in FIG. 1 , receivingstreaming image captures from imaging device 410. In some embodiments,imaging device 410 may be an endoscope and may be controlled by anarticulated arm, such as articulated arm 102 of FIG. 1 and/or thearticulated arm of FIG. 2 . In FIG. 4A, camera view 400 is delineated bythe dotted line which may represent an exemplary field of view and focusarea for imaging device 410. In FIG. 4B, an exemplary camera view 400 isshown from the perspective of a user viewing a video stream from imagingdevice 410 on a display, such as display system 192 of operatorworkstation 190 of FIG. 1 . In some embodiments, the video streamprovided by imaging device 410 may be stereoscopic. Imaging device 410may use one or more sensors for providing stereoscopic video streams. Inthis manner, the operator may have a sense of depth perception whenusing a system such as computer aided system 100 of FIG. 1 . Cameracoordinate frame 411 illustrates the coordinate frame of imaging device410. In FIG. 4A, camera coordinate frame 411 shows the Z1 and X1 axes ofcamera coordinate frame 411 with the Y1 axis (not shown) going in andout of the page. In FIG. 4B the Y1 and X1 axes of camera coordinateframe 411 are shown with the Z1 axis (not shown) going in and out of thepage. In some embodiments, camera coordinate frame 411 may be cameracoordinate frame 363 of FIG. 3 .

FIGS. 4A and 4B also include instruments 420 and 430, which may also becontrolled by one or more articulated arms, such as articulated arms 102of FIG. 1 and/or the articulated arm of FIG. 2 . Instruments 420 and 430may be within camera view 400 and may be manipulated by one or moreusers or operators using controls, such as input controls 195 of FIG. 1, and viewing instruments 420 and 430 from the perspective of FIG. 4B.FIGS. 4A and 4B also illustrate coordinate frames 421 and 431 ofinstruments 420 and 430, respectively, from different perspectives. Insome examples coordinate frames 421 and 431 may be the same asinstrument coordinate frames 343 and 353 of FIG. 3 .

Because a user controlling instruments 420 and 430 is viewing theinstruments from the perspective of FIG. 4B of camera view 400, it maybe useful for user commands to be conducted in the camera coordinateframe 411. Any commands provided in the camera coordinate frame 411 aretranslated to commands in the coordinate frames 421 and 431 of theinstruments, by using a kinematic model, such as kinematic model 300 ofFIG. 3 . In this manner, up and down may be in relation to the cameraview, which may be generally in line of the perspective of the user. Auser command to move instrument 420 or 430 up and down may translate tothe instrument moving along the Y1 axis of camera coordinate frame 411.Similarly, user commands for other translational motions may follow theX1 and Z1 axes of camera coordinate frame 411. In some embodiments,commands for rotational motions, such as roll, pitch, and yaw, may alsobe translated from the camera coordinate frame 411 to the coordinatereference frames 421, 431 of the instruments.

In some embodiments, camera coordinate frame 411 may be detached fromthe physical imaging device 410. For example, camera coordinate frame440 may be a saved camera coordinate frame for a prior position ofimaging device 410 before a movement 441 of imaging device 410. Theposition of camera coordinate frame 440 may be stored on a computerreadable medium, such as memory 150 of FIG. 1 . In some embodiments,coordinate frame 440 may be stored as a transformation from a referencecoordinate frame, such as arm mounting platform coordinate frame 330,Table top coordinate frame 310, remote center of motion coordinate frame342, and/or the like of FIG. 3 . In some embodiments, camera coordinateframe 440 may be stored as a configuration of an articulated arm basedon the position of one or more joints.

Having camera coordinate frame 440 detached from the actual camera maybe beneficial when instrument motion is determined in relation tocoordinate frame 440. For example, if the position of instruments 420and 430 were commanded in relation to camera coordinate frame 411 andcamera coordinate frame 411 was fixed to imaging device 410, undesirabledisturbances to imaging device 410 would translate into undesirabledisturbances to instruments 420 and 430. Instead, if the position ofinstruments 420 and 430 are commanded in relation to a saved coordinateframe, such as camera coordinate frame 440, this problem would not occurbecause camera coordinate frame 440 is a saved coordinate frame that ispotentially independent of any movement of imaging device 410.

In some embodiments, it may be beneficial to latch camera coordinateframe 440 relative to a reference frame, such as table top coordinateframe 310, remote center of motion coordinate frame 342, and/or the likeof FIG. 3 . Camera coordinate frame 440 may be latched relative to areference coordinate frame such that when the reference frame moves,camera coordinate frame 440 will also move and maintain distance,orientation, and/or position in relation to the reference coordinateframe. For example, if camera coordinate frame 440 were latched relativeto table top coordinate frame 310 of FIG. 3 , when the table topphysically moved up (moving the table top coordinate frame), coordinateframe 440 would also move up. Similarly, if the table top rotated,camera coordinate frame 440 may also rotate along the same axis as thetable. As will be discussed in more detail below, this is beneficialwhen allowing for table movement. In some embodiments, it may beadvantageous to latch camera coordinate frame 440 to the table topcoordinate frame during table motion because there may be situationwhere table motion does not translate to movements in another coordinateframe, such as remote center of motion coordinate frame 342.

In some embodiments, a user may have the option to move and/or realignthe camera coordinate frame 440 with the imaging device 410 and cameracoordinate frame 411. In this manner when imaging device 410 strays toofar from camera coordinate frame 440 such that instrument movementsbecome less intuitive to the user, the user may reset camera coordinateframe 440.

As described previously, as a computer-assisted device, such ascomputer-assisted devices 110 and/or 210, is being operated it would bedesirable to allow continued control of the instrument and/or endeffectors while motion of a surgical table, such as surgical tables 170and/or 280, is allowed. In some examples, this may allow for a lesstime-consuming procedure as surgical table motion may occur withouthaving to remove instruments that are inserted into the patient. In someexamples, this may allow a surgeon and/or other medical personnel tomonitor organ movement while the surgical table motion is occurring toobtain a more optimal surgical table pose. In some examples, this mayalso permit active continuation of a surgical procedure during surgicaltable motion.

FIG. 5 is a simplified diagram of an exemplary method 500 formaintaining intuitive controls for an end effector on an articulated armwhile one or more joints of the articulated arm by being moved by anexternal force. According to some embodiments, method 500 may includeone or more of the processes 510-595 which may be implemented, at leastin part, in the form of executable code stored on a non-transitory,tangible, machine readable media that when run on one or more processors(e.g., the processor 140 in control unit 130 of FIG. 1 ) may cause theone or more processors to perform one or more of the processes 510-595.

At process 510, a coordinate frame and/or configuration for an imagingdevice may be stored in a memory, such as memory 150 of FIG. 1 ,creating a virtual coordinate frame. The virtual coordinate frame may bestored in any suitable data structure capable of representing ahomogenous transform, such as in the form of one or more matrixes,and/or the like. In some examples, the virtual coordinate frame may bestored in memory in the form of a transform based on a kinematic modelfor an articulated arm controlling the imaging device. In someembodiments, the virtual coordinate frame may be stored as joint angles,positions and/or configurations on the articulated arm from which thetransform may be recomputed using one or more kinematic models of thearticulated arm. In some embodiments, the virtual coordinate frame maybe stored in memory as a position and orientation in relation to areference coordinate frame, such as arm mounting platform coordinateframe 330 and/or device base coordinate frame 320 of FIG. 3 . In someembodiments, the virtual coordinate frame may be based on a position ofan imaging device at a certain point in time, such as before a brakerelease or before one or more joints for one or more articulated arms isset to a floating state. Joints set to a floating state may generally bereferred to as floating joints. In some embodiments, multiple copies ofthe virtual coordinate frame may be stored, such that each virtualcoordinate frame may be modified in relation to multiple situations fora particular articulated arm and/or for different articulated arms. Forexample, there may be several articulated arms controlling severalinstruments within view of a single imaging device. Each of the severalarticulated arms may be positioned in a different manner and there maybe one or more virtual camera coordinate frames stored in memory foreach articulated arm and in relation to an articulated arm, such as theremote center of motion of the articulated arm. These virtual cameracoordinate frames may be modified based on motions to one or more of thearticulated arms.

At process 520, one or more joints making up a floating joint set forthe articulated arm holding an end effector and/or instrument (hereinafter “instrument arm”) may be put into a floating state. When jointsare set in a floating state, the floating joints may move freely uponthe introduction of a force. In some embodiments, the articulated armfor an imaging device (hereinafter “imaging arm”) may also have a jointset that is put into a floating state. The floating joints, when in afloating state, may be gravity compensated to prevent and/or reducemovements to one or more joints from the force of gravity. In accordancewith FIG. 2 , set up joints 240 may make up the floating joint set(s)for the instrument arm and/or imaging arm.

In some examples the floating joint set(s) may receive physical motionfeedback. In some examples the physical motion feedback may beindirectly caused by a device, such as surgical table 280 of FIG. 2 .For example, a remote center of motion of an articulated arm may beplaced at a body opening of a patient. The patient may be resting on thesurgical table such that when the table moves, the patient is moved. Themovement of the patient may cause the body opening of the patient tomove, which indirectly causes movement to the articulated arm. Thisforce, in turn, may cause one or more of the floating joints in thejoint set(s) to move. This is sometimes referred to as instrumentdragging.

However, the floating joints may receive motion feedback from othersources. In some examples, the source of the physical motion feedbackmay be from an operator pushing and/or pulling on the articulated arm.For example, an operator may pull on the remote center of motion of anarticulated arm causing the floating joint set(s) to receive physicalmotion feedback. In some examples, an operator may be helping to steadyone or more articulated arms in connection to a patient while thepatient is being moved. One of ordinary skill in the art will recognizethat there are several sources of physical motion feedback that mayresult in changes to the positions and/or orientations of the floatingjoints, all of which are contemplated herein.

In the following examples for method 500, the physical motion feedbackis discussed in relation to a moving surgical table that iscommunicatively connected and/or registered in some manner with thearticulated arm. For example, in FIG. 1 , surgical table 170 iscommunicatively coupled to arms 120 through control unit 130 and device110. However, one of ordinary skill in the art will recognize thatmethod 500 may be applied to any situation where an object directlyand/or indirectly moves the articulated arm independent of whether theobject is communicatively connected and/or registered in some mannerwith the articulated arm. For example, any process in method 500 thatuses some communication from the surgical table may be ignored/omittedwhen method 500 is applied for objects that are not registered.Furthermore, instead of the articulated arms being moved by the surgicaltable, the articulated arms may be moved by other disturbances.

At process 530, an object, such as a surgical table, may directly and/orindirectly move one of the floating joints of the instrument arm and/orimaging arm. In some embodiments, the surgical table may have severaldegrees of freedom for movement, such as up/down, left/right,forward/backwards, tilt, Trendelenburg, slide, roll, pitch, yaw, and/orthe like. The table movement may be controlled by a controller, such ascontrol unit 130 of FIG. 1 . In some embodiments, the controller may becontrolling the movement of the table based on user input. In someexamples the controller may move the table based on a preprogrammedtable movement. In some embodiments, the surgical table may beconfigured to move a patient that is resting on the table. Though theexample above is a surgical table that causes a direct and/or indirectmotion to the floating joints, in other embodiments, other devicesand/or objects may be causing the direct and/or indirect motion to thearticulated arm(s).

At process 540, motion of one or more floating joints of the instrumentarm and/or imaging arm caused by the table movement in process 530 isdetected. In some embodiments, the instrument arm and/or imaging arm mayplace the remote center of motion of the instrument arm and/or imagingarm at a patient body opening. The movements in process 530 may move theremote center of motion of the instrument arm and/or imaging arm, whichin turn disturbs one of the floating joints of each arm. In someembodiments, the movement of the remote center of motion may beindirect, such as the surgical table moving a patient which in turncauses instrument dragging. A control unit, such as control unit 130,receives indicators that one or more of the floating joints in theinstrument arm and/or imaging arm has moved. The indicators may comefrom sensors in the floating joints providing joint positioning data.

In some embodiments, the detected motion is transformed from a localcoordinate frame to the virtual coordinate frame created during process510. In some embodiments, the transformed motion may be separated intodifferent components such as translational and rotational motions in thevirtual coordinate frame.

At process 550, motion readings, data, and/or reports from the surgicaltable are received and transformed into the virtual coordinate frame.The transformed motion may be separated into the translational androtational components once transformed into the virtual coordinateframe. For example, in accordance with FIG. 3 , table top coordinateframe 310 may change based on the movement of the table. The changes ormovement may be transformed from coordinate frame 310 to anothercoordinate frame, such as remote center of motion coordinate frame 342.Transforming the movement from reference frame of table top coordinateframe 310 to remote center of motion coordinate frame 342, in accordanceto FIG. 3 , is determined through transforms 325, 344, and 345. Themovement of the table are then transformed from the coordinate frame ofthe remote center of motion to the virtual coordinate frame (not shown)using the transform stored at process 510.

In some embodiments, this process of method 500 may be omitted. Forexample, process 550 may be omitted when the surgical table, or otherobject, does not provide and/or report motion readings or providesinaccurate readings.

In some embodiments, process 540 or 550 may be omitted when suitablemotion data may be derived from one or the other motion/positioningdata. For example, the surgical table may have perfect registration withrespect to the instrument arm and/or imaging arm. In some examples, whenthe surgical table has the same degrees of freedom or motion as thefloating arms, the joint positioning data may be sufficient.

At optional process 560, the instrument arm may move a point of interestof the instrument arm, such as the remote center of motion and/or apoint on an end effector, using a second set of joints, which may bedifferent from the floating joints, to track and compensate for themotion of the surgical table. These second set of joints may also bereferred to as compensating joints of the instrument arm. Compensatingjoints may generally refer to joints which may be driven or maycompensate for other motion. In some cases, the compensating joints maybe mutually exclusive from floating joints. Several compensating jointsmay be referred to as a set of compensating joints and/or as acompensating joint set. This may be beneficial when the floating jointslack one or more degrees of motion in relation to the surgical table.For example, the floating joint set may be unable to rotate and/ortranslate the remote center of motion, instrument, and/or end effectorto maintain orientation with respect to the surgical table and/orpatient when the surgical table is moved. When an instrument arm cannotmove in relation to a patient being moved, the instrument shaft and/orend effector may move relative to the patient causing injury to thepatient and/or the instrument.

In some examples, the second joint set may compensate for the lack ofmotion in the floating joints by transforming table motion data from thetable coordinate frame to the coordinate frame of the point of interest(transformed motion data), isolating the portion of the transformedmotion data that the floating joints cannot move, such as rotationalmotion, and driving the second joint sets to move based on the isolatedportion of the translated motion data.

For example, in accordance with FIG. 3 , movement in table topcoordinate frame 310 is transformed to remote center of motioncoordinate frame 342. The transformed movement (in the remote center ofmotion coordinate frame 342) that the floating joints cannot compensatefor is separated from the transformed movement. For example, thetransformed movement may be represented as matrix. When the floatingjoints can translate the remote center of motion but not rotate remotecenter of motion, the portion of the matrix that representstranslational movements is zeroed out and the rotational portions of thematrix left alone. This, in effect, isolates the transformed movement tojust the rotational movement. The joints are then driven to move theremote center of motion based on the isolated transformed movement.

Similarly the imaging arm may also move a point of interest of theimaging arm, such as the imaging device and/or a remote center ofmotion, using a second joint set to track the motion of the surgicaltable. The method would be the same as with the instrument arm, exceptmotions would be transformed to a coordinate frame related to theimaging device, for example, remote center of motion coordinate frame352. In this manner, the instruments of the instrument arm maintaintheir orientation to the table and the imaging device of the imagingarm. As a result, an operator viewing the instruments through imagesprovided by the imaging device would see patient anatomy moving, but theinstrument, camera, and patient move in a fixed manner with the tabletop may appear stationery and/or relatively stationary. This is becausethe entire camera and instrument may be moving in line or very close toin line with each other and/or the table top.

At process 570 the virtual coordinate frame is moved in relation to themovement of the instrument arm. In some embodiments the virtualcoordinate frame is moved in relation to a point of interest on theinstrument arm, such as a remote center of motion.

In some examples, the virtual coordinate frame is linked to a point ofinterest, such as a remote center of motion of the instrument arm. Inthis manner, as the point of interest of the instrument arm is moved bythe table and/or moved to compensate for the table motion. The virtualcoordinate frame is moved in relation to the point of interest as ifthere were a rigid object connecting the virtual coordinate frame withthe remote center of motion.

In some examples, the positioning of the virtual coordinate frame ismoved to track the point of interest using the joint motion data. Forexample, the virtual coordinate frame is moved based on the transform ofthe joint positioning data at process 540 to the coordinate frame of thevirtual coordinate frame.

Similarly, the positioning of the virtual coordinate frame is moved totrack the point of interest using the surgical table motion data. Forexample, the virtual coordinate frame is moved based on the transform ofthe table motion data at process 550 to the coordinate frame of thevirtual coordinate frame.

In some embodiments, the virtual coordinate frame may be moved using acombination of the joint motion data and table motion data. For example,the joint motion data of process 540 transformed to the virtualcoordinate frame, and table motion of process 550 that is transformed tothe virtual coordinate frame and isolated for degrees of motions lackingin the floating arms may be used. In some examples the isolated motionsfrom the table motion may be rotational movement in the virtualcoordinate frame.

In some embodiments, the virtual coordinate frame is moved in relationto the movement of another point of interest, such as a table top. Thevirtual coordinate frame uses the table motion data to change theposition of the virtual coordinate frame in relation to the table top.This may be useful when the table can substantially change inorientation without inducing appreciable movement of the remote centersof motion of an articulate arm. For example, isocentered motion.

By moving virtual coordinate frame based on the movement of the tableand/or floating joints, the virtual coordinate frame maintains anaccurate predicted position of the coordinate frame for the imagingdevice. Because a user manipulating the instruments may be viewing theinstruments through the imaging device and the virtual coordinate frameis a good predictor of the position of the imaging device, a user maynot be able to tell the difference between instrument motion commandsapplied using the virtual coordinate frame and instrument motioncommands applied using the actual coordinate frame of the imagingdevice. This effectively approximates instrument motion commands in thereference frame of the imaging device, even when the imaging device isnot perfectly positioned and/or oriented with the virtual coordinateframe.

However, in cases where the virtual coordinate frame has strayed too farfrom the imaging device, such that a user can tell that the commands areno longer in the coordinate frame of the imaging device, the user may beable to reset the position of the virtual coordinate frame to the actualposition of the imaging device.

At process 580, commands to move an instrument and/or end effector ofthe instrument arm may be received. The instrument motion commands maycome from a user manipulating input controls, such as input controls195. In some examples, the instrument motion commands may be receivedduring table motion.

At process 590, the commands at 580 are transformed from movementcommands in the virtual camera coordinate frame to the coordinate frameof the instrument and/or end effector using a kinematic model, such asthe kinematic model of FIG. 3 . In some examples, compensating jointchanges for the instrument arm are determined based on the movementcommands in the coordinate frame of the instrument and/or end effector.In some examples, the movement commands in the coordinate frame of theinstrument and/or end effector are mapped from the coordinate frame ofthe end effector to one or more local coordinate systems associated witheach of the compensating joints. In effect, this transforms the motioncommands from the virtual camera coordinate frame to the instrumentand/or end effector coordinate frame and then from the end effectorcoordinate frame to movements commands relative to the compensatingjoints of the instrument arm. In some examples, one or more kinematicmodels are used to transform the movement commands to the each of thecoordinate frames. In some examples, the compensating joints may includeany of the joints of the articulated arm and/or the manipulator that arenot part of the first joint set. Once the movement commands for therespective coordinate frames of each joint are determined, they are usedto determine the movements for each of the compensating joints. In someexamples, an inverse Jacobian may be used to map the movement commandsto movements of the compensating joints. In some examples, the movementsin the compensating joints may be applied as joint velocities applied tothe compensating joints.

At process 595, the second set of joints is driven to move theinstrument and/or end effector in accordance with the transformedcommands determined at process 580. In some embodiments, the instrumentsmay be moved and/or driven while the table is in motion. According tosome embodiments, one or more of the processes 510-590 may be performedconcurrently. For example, user commands driving the joints aresuperimposed onto the commands to drive the joints based on the tablemotions such that while the table moves, the floating joints are movedby the table, the compensating joints may move in accordance withprocess 560 while the compensating joints also are moving in accordancewith process 580-590.

As discussed above and further emphasized here, FIG. 5 is merely anexample which should not unduly limit the scope of the claims. One ofordinary skill in the art would recognize many variations, alternatives,and modifications. According to some embodiments, one or more of theprocesses 510-595 may be performed in different orders than the orderimplied in FIG. 5 . In some examples, one or more of the process 510-595may be performed in any order and/or partially or totally in parallel.In some embodiments, one or more process 510-595 may be omitted/ignored.

FIG. 6 is a simplified diagram of method 600 for maintaining intuitivecontrols for an end effector on an articulated arm during table motion.In some examples a user may be operating one or more end effectors ofone or more articulated arms using a controller. In some examples, theend effectors on the one or more articulated arms may be end effectorson articulated arms 120 of FIG. 1 . In some example, the user may becontrolling the end effectors and the articulated arms through operatorworkstation 190 of FIG. 1 . In some examples, the user may be viewingthe one or more end effectors through a display system 192 which may bedisplaying image captures from an image device on an articulated arm.

At process 610, user controls are set up such that control commands tomove an end effector may be conducted in relation to a virtualcoordinate frame of an imaging device. In some embodiments the virtualcoordinate frame is created using process 510 of FIG. 5 . In someexamples, received user commands are transformed from movement commandsin the virtual camera coordinate frame to the coordinate frame of theinstrument and/or end effector using a kinematic model, such as thekinematic model of FIG. 3 .

At process 620, a surgical table is moved, which may directly and/orindirectly physically move a patient resting on the table, thearticulated arms, the end effectors, and/or the imaging device. Thesurgical table movement may be in response to user commands to translateand/or rotate the table. In some embodiments, the articulated arms mayhave a set of joints that are in a floating state such that the tablemay directly and/or indirectly move the floating joints.

At process 630, the end effectors, imaging device and/or the virtualcoordinate frame are moved to maintain relative distance and orientationwith the table top of the surgical table as the table moves. Forexample, if the table top moves up, at process 630 the end effectors,imaging device and the virtual coordinate frame also moves up. Also, ifthe table top rotates, the imaging device, end effectors, and virtualcoordinate frame are rotated along the same axis as the axis of thetable top. In this manner, to a user viewing images captured by theimaging device, the user would perceive anything not maintainingorientation with the table top during table motion (e.g. unsecuredpatient anatomy) as moving. In contrast, the user would perceive thetable top and objects either constrained to the table top or maintainingorientation with the table top as stationary.

In some examples, the movements to the end effector and imaging devicemay be approximations through instrument dragging. In some examples themovements to the end effector and imaging device may be compensationmovements of one or more joint to maintain relative position of an endeffector in relation to the table top during table motion. In someexamples, movement of the virtual coordinate frame is based on tableand/or table top motion data during table motion. For example, thevirtual coordinate frame may be latched in relation to the table top,and as the table top is moved, the virtual coordinate frame is alsomoved to maintain orientation and position with the table top. Process630 may implement one or more processes of FIG. 5 such as processes540-570.

At process 640, user controls and commands are continually configured tomove the end effector in relation to the virtual coordinate frame as thevirtual coordinate, end effector, and/or imaging device moves accordingto process 630 with the surgical table as the surgical table is movedaccording to process 620. In some embodiments, process 640 may beimplemented using processes 590-595 of FIG. 5 .

As discussed above and further emphasized here, FIG. 6 is merely anexample which should not unduly limit the scope of the claims. One ofordinary skill in the art would recognize many variations, alternatives,and modifications. According to some embodiments, one or more of theprocesses 610-640 may be performed in different orders than the orderimplied in FIG. 6 . In some examples, one or more of the process 610-640may be performed in any order and/or partially or totally in parallel.In some embodiments, one or more process 610-640 may be omitted/ignored.

Some examples of control units, such as control unit 130 may includenon-transient, tangible, machine readable media that include executablecode that when run by one or more processors (e.g., processor 140) maycause the one or more processors to perform the processes of method 500.Some common forms of machine readable media that may include theprocesses of method 500 are, for example, floppy disk, flexible disk,hard disk, magnetic tape, any other magnetic medium, CD-ROM, any otheroptical medium, punch cards, paper tape, any other physical medium withpatterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chipor cartridge, and/or any other medium from which a processor or computeris adapted to read.

Although illustrative embodiments have been shown and described, a widerange of modification, change and substitution is contemplated in theforegoing disclosure and in some instances, some features of theembodiments may be employed without a corresponding use of otherfeatures. One of ordinary skill in the art would recognize manyvariations, alternatives, and modifications. Thus, the scope of theinvention should be limited only by the following claims, and it isappropriate that the claims be construed broadly and in a mannerconsistent with the scope of the embodiments disclosed herein.

What is claimed is:
 1. A computer-assisted device comprising: a firstarticulated arm, the first articulated arm being configured to supportan end effector; and a control unit coupled to the first articulatedarm; wherein the control unit is configured to: determine a virtualcoordinate frame, the virtual coordinate frame being of an imagingdevice for capturing images of a workspace of the end effector, and thevirtual coordinate frame being detached from an actual imagingcoordinate frame of the imaging device; receive, from an input controlconfigured to be manipulated by a user, a first instrument motioncommand to move the end effector; and drive the first articulated arm tomove the end effector relative to the virtual coordinate frame based onthe first instrument motion command.
 2. The computer-assisted device ofclaim 1, wherein the virtual coordinate frame is based on a pose of theimaging device before one or more first joints in a second articulatedarm are set to a floating mode, the second articulated arm beingconfigured to support the imaging device.
 3. The computer-assisteddevice of claim 2, wherein a movement of a table causes a movement ofthe imaging device while the one or more first joints are in thefloating mode, the table being separate from the computer-assisteddevice.
 4. The computer-assisted device of claim 3, wherein the controlunit is further configured to move, based on motion data received fromthe table and using one or more second joints of the second articulatedarm, the imaging device to maintain a relative distance and a relativeorientation between a top of the table and the imaging device.
 5. Thecomputer-assisted device of claim 3, wherein the movement of the imagingdevice is caused by a movement of a patient into which the imagingdevice is inserted, the patient being located on the table, and themovement of the patient being due to the movement of the table.
 6. Thecomputer-assisted device of claim 1, wherein the control unit is furtherconfigured to, prior to determining the virtual coordinate frame:receive, from the input control, a second instrument motion command tomove the end effector; and drive the first articulated arm to move theend effector relative to the actual imaging coordinate frame based onthe second instrument motion command.
 7. The computer-assisted device ofclaim 1, wherein the control unit is further configured to maintain thevirtual coordinate frame in fixed relationship to a top of a table basedon motion data received from the table, the table being separate fromthe computer-assisted device.
 8. The computer-assisted device of claim1, wherein the control unit is further configured to realign the virtualcoordinate frame with the imaging device.
 9. The computer-assisteddevice of claim 1, wherein the control unit is further configured to:reset, in response to a command received from the user, a position ofthe virtual coordinate frame to an actual position of the imaging deviceto generate an updated virtual coordinate frame; and receive, from theinput control, a second instrument motion command to move the endeffector; and drive the first articulated arm to move the end effectorrelative to the updated virtual coordinate frame based on the secondinstrument motion command.
 10. The computer-assisted device of claim 1,wherein the control unit is further configured to: configure a firstjoint of the first articulated arm to a floating mode; detect movementof the first joint; and drive a second joint of the first articulatedarm based on the movement of the first joint.
 11. The computer-assisteddevice of claim 10, wherein the control unit is further configured to:determine movement of a table, the table being separate from thecomputer-assisted device; and drive the second joint further based onthe movement of the table.
 12. A method for operating acomputer-assisted device, the computer-assisted device comprising afirst articulated arm configured to support an end effector, the methodcomprising: determining, by a control unit, a virtual coordinate frame,the virtual coordinate frame being of an imaging device for capturingimages of a workspace of the end effector, and the virtual coordinateframe being detached from an actual imaging coordinate frame of theimaging device; receiving, by the control unit from an input controlconfigured to be manipulated by a user, an instrument motion command tomove the end effector; and driving, by the control unit, the firstarticulated arm to move the end effector relative to the virtualcoordinate frame based on the instrument motion command.
 13. The methodof claim 12, wherein: the virtual coordinate frame is based on a pose ofthe imaging device before one or more first joints in a secondarticulated arm are set to a floating mode, the second articulated armbeing configured to support the imaging device; and a movement of atable causes a movement of the imaging device while the one or morefirst joints are in the floating mode, the table being separate from thecomputer-assisted device.
 14. The method of claim 13, further comprisingmoving, by the control unit based on motion data received from the tableand using one or more second joints of the second articulated arm, theimaging device to maintain a relative distance and a relativeorientation between a top of the table and the imaging device.
 15. Themethod of claim 12, further comprising maintaining, by the control unit,the virtual coordinate frame in fixed relationship to a top of a tablebased on motion data received from the table, the table being separatefrom the computer-assisted device.
 16. The method of claim 12, furthercomprising: resetting, by the control unit in response to a commandreceived from the user, a position of the virtual coordinate frame to anactual position of the imaging device to generate an updated virtualcoordinate frame; and receiving, by the control unit from the inputcontrol, a second instrument motion command to move the end effector;and driving, by the control unit, the first articulated arm to move theend effector relative to the updated virtual coordinate frame based onthe second instrument motion command.
 17. The method of claim 12,further comprising: configuring, by the control unit, a first joint ofthe first articulated arm to a floating mode; detecting, by the controlunit, movement of the first joint; and driving, by the control unit, asecond joint of the first articulated arm based on the movement of thefirst joint.
 18. A non-transitory machine-readable medium comprising aplurality of machine-readable instructions which when executed by one ormore processors associated with a computer-assisted device, are adaptedto cause the computer-assisted device to perform a method comprising:determining a virtual coordinate frame, the virtual coordinate framebeing of an imaging device for capturing images of a workspace of an endeffector, and the virtual coordinate frame being detached from an actualimaging coordinate frame of the imaging device; receiving, from an inputcontrol configured to be manipulated by a user, an instrument motioncommand to move the end effector; and driving a first articulated arm ofthe computer-assisted device to move the end effector relative to thevirtual coordinate frame based on the instrument motion command, thefirst articulated arm being configured to support the end effector. 19.The non-transitory machine-readable medium of claim 18, wherein: thevirtual coordinate frame is based on a pose of the imaging device beforeone or more first joints in a second articulated arm are set to afloating mode, the second articulated arm being configured to supportthe imaging device; and a movement of a table causes a movement of theimaging device while the one or more first joints are in the floatingmode, the table being separate from the computer-assisted device. 20.The non-transitory machine-readable medium of claim 19, wherein themethod further comprises: moving, based on motion data received from thetable and using one or more second joints of the second articulated arm,the imaging device to maintain a relative distance and a relativeorientation between a top of the table and the imaging device; ormaintaining the virtual coordinate frame in fixed relationship to thetop of the table based on the motion data received from the table.